Field of the Invention
This invention relates generally to the field of nanofabrication and, more specifically, to the construction of synthetic neurons.
Description of the Related Art
Artificial Neural Networks (ANN) are biology-inspired computer systems capable of addressing problems having a large number of unknown inputs. These systems are trained by repetition, with each iteration reinforcing specific neural pathways until an approximately suitable solution can be found. With the growth of computer-assisted decision making, ANN systems are increasingly at the heart of a wide variety of fields including search engines, voice recognition, vehicular/robotic automation and financial trading systems.
The vast majority of existing ANN are software emulations. Commercial state of the art Graphics Processor Units (GPU) now allow for the implementation of deep neural networks for visual recognition. However, in the most powerful type of ANN, Recurrent Neural Networks (RNN), the state of each neuron and connection has to be calculated with respect to other neurons and connections of the system. This updating process therefore creates a system complexity that increases exponentially with an increasing number of neurons. Moreover, since this process must be performed iteratively in software, it limits the potential size and speed of the ANN. Thus, while representing a significant advance in ANN technology, the GPU approach suffers from an inherent limitation in its simulation capabilities.
Another methodology used in the past is the building of a “physical” neural network where physical relationships, as opposed to software algorithms, emulate neuronal processes. This removes the need to iteratively calculate every update in the state of the neurons. By avoiding these calculations, the regression imposed by the learning process becomes a one-step operation, independent of the size of the neural network.
One of the key aspects of much of the prior art is based on the concept of a “memristor”. A memristor is a device whose electrical resistance depends on how much electric charge has flowed through it in a given direction in the past. That is, the device effectively remembers its history. Prior to 2007, such a device did not exist in nanoelectronics, but was composed of macroscopic capacitors and variable resistances. In 2007 researchers presented an integrated solid-state TiO2-based memristor in which thin double layers of TiO2 films are encapsulated in metal electrodes. One of the TiO2 layers is depleted of oxygen atoms. With the application of an electric field, oxygen vacancies drift, and thus change the resistance of the layers. In 2014, a solution consisting of MoOx/MoS2 and WOx/WS2 heterostructures sandwiched between two printed silver electrodes was demonstrated to be a potential candidate for memristive and memcapacitive switches in printable electronics. Both of these developments provide fast ion conduction at nanoscale and are considered nanoionic devices with great potential in the field of flash memory.
A fundamental concept in ANN is the leaky integrate-and-fire neuron, which is recognized by those knowledgeable in the field to be a formal spiking neuron model. The general operation of this neuron is analogous to its biological counterpart, that is, the neuron integrates many input signals from other neurons in a nonlinear fashion and fires a signal pulse if the sum of the inputs reaches a threshold. This model presents one of the best analogies to biological neurons and can work dynamically (as a function of time) in RNN, where connections between units can form directed cycles (as in the brain).
A three-terminal implementation of a memristor is called a memistor. Such a device is similar to a transistor, where the conductance between two of the terminals is controlled by the third one. However, as opposed to a transistor which operates instantaneously, the memistor integrates the current over time in the third terminal. This adds a dynamic component to the system.
Early physical neural networks include ADALINE in the 1960's, which is a single-layer artificial neural network device implemented using assembled memistor systems. More recent progress has been presented by DARPA SyNAPSE project, which includes an approach based on TiO2 memristor technology as described above.
Another approach to SyNAPSE is to use classical transistors to emulate neural behaviors.
A limitation of current memistors is the requirement of connecting them with other nanodevices to form an artificial neuron. As mentioned above, an activated memistor stays in its current state and has no dynamic responses (thus a good application for flash memory). It serves as an accumulation unit, but cannot intrinsically generate the necessary neuron weightings and is incapable of pulsing a signal output. Therefore, memistors cannot operate as the leaky integrate-and-fire neurons required in dynamic RNN without additional nanocomposites (e.g., integrated capacitances and/or inductances).
A different type of memistor is proposed by U.S. Pat. No. 6,999,953 and is based on phase changing material consisting of In, Ag, Te, Se, Ge, Sb, Bi, Pb, Sn, As, S, P, and mixtures or alloys thereof. The patent claims an analog neuron composed of three modules of phase changing material: a weighting unit, an accumulation unit and an optional activation unit connected in series. As in the case of TiO2 memistor, this device is not dynamic and requires other nanodevices (e.g., capacitances) to build an RNN. The system also presents significant drawbacks over other memistors: various modules are required to produce the memistor effect, and it is made of complex alloys which are both expensive to manufacture and difficult to manipulate. Another problem with these types of composite materials is their very high activation energy, requiring temperatures in excess of 500° K, which are very impractical in micro-nanoelectronics.
More recently, researchers from Pennsylvania State University have proposed using the synchronized charge oscillations in correlated electron systems as a potential candidate for oscillator-based non-Boolean computing. This work involves the electrical connection of two macroscopic vanadium dioxide (VO2) devices coupled through a capacitance. As the scale is macroscopic, however, the density of the devices is inherently limited. Researchers at the Institut National de Research Scientifique (INRS) at Varennes, Quebec are also working with VO2, and have reported the production of optical switches using this material (see, e.g., M. Soltani, et al., 1×2 optical switch devices based on semiconductor-to-metallic phase transition characteristics of VO2 smart coatings, Meas. Sci. Technol. 17 (2006) 1052-1056), as well as well as a system for generating a negative capacitance (described in U.S. Patent Application Publication No. 2012/0286743).